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A method for detecting and quantifying mo- 
tion of a body part of an individual or animal, such 
as an individual's face or portion thereof. A digitized 
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(changed image). The referenced image is compared 
with the changed image to determine movement par- 
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METHOD AND APPARATUS FOR DETECTING AND QUANTIFYING 
MOTION OF A BODY PART 

5 FIELD OF THE INVENTION 

The present invention relates generally to methods for 
detecting and quantifying motion of a body part, and more 
particularly, but not by way of limitation, to a method and 
apparatus for detecting and quantifying motion of a body 
10 wherein a digitized image of a body part in a control position 

is produced and a digitized image of the body part in a 
changed position is produced and the two images are compared 
to determine movement of the body part between the two 
digitized images. 
15 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagrammatic, schematic view of the system 
of the present invention. 

Figure 2 is a flow chart diagrammatic illustrating the 
program used in connection with the processor shown in Figure 
20 1- 

Figure 3 is a diagrammatic view of . an individual's or 
patient's face to be analyzed using the system of Figure 1, 
and showing grid lines and areas of interest used in the 
system of Figure 1 and showing such grid lines and areas of 
25 interest as they would appear on the screen of the computer 

monitor portion of the system shown in Figure 1. 

Figure 4 is a diagrammatic view of the face of Figure 3 
with marker dots placed on the face for use in aligning frames 
to be compared which are used in one embodiment of the present 
30 invention. 

Figure 5 is a pictorial view of the face of Figures 3 and 
4 showing the individual's head located in a head holder. 

Figure 6 is a side view of an individual with its head 
disposed in the head holder. 
3 5 Figure 7 is a partial perspective view of the head holder 

of Figures 5 and 6. 
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unpleasant odor, not only is the expected nears movement 
noted, but additional accessory strategies which vary 
considerably among individuals may be observed- The present 
computer analysis system permits the separation of the 
accessory strategy movement from the requested movement and 
the analysis of these two movements in combination or 
separately. 

The present system can separate brow movement into at 
least three components (forehead wrinkling, eyebrow movement 
and upper eyelid movement) . The present system also can 
analyze these movements individually or collectively. 

The system of the present invention can identify three 
components of eye closure (upper eye lid movement, lower eye 
lid movement, and surrounding orbicularis oculi movement) and 
analyze these components individually or collective. 

The system of the present invention also can detect 
several components of smiling at cites around the mouth and 
distant from it. For example, movement of the commissure, 
movement of the nasolabial fold, movement of the eye and upper 
cheek, movement along the distribution of the zygomaticus 
major. 

The system of the present invention can identify and 
measure movement in secondary areas to the primary area of 
requested movement thereby allowing quantitative measurement 
of abnormal facial movement, such as synkinesis. 

It should be noted that, although the system of the 
present invention has been described herein, in some 
instances, with specific reference to detecting and 
quantifying facial movements, the system of the present 
invention is useable for detecting and quantifying motion of 
any body part. The references herein to facial movements are 
not to be construed as limiting the present invention to 
facial movement. The present invention also is useful for 
analyzing motion of a body part of an animal. 

Shown in Figure 1 is a diagrammatic, schematic view of a 
motion analysis apparatus 10 which is constructed in 
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accordance with the present invention. The motion analysis 
apparatus lo includes an image producing means 12 which is 
constructed and adapted to produce a digitized image of the 
body part and to output the digitized image over a signal path 
14. The digitized image outputted over the signal path 14 is 
inputted into a processor 16. The processor 16 processes the 
digitized image in a manner to be described in greater detail 
herein, 

A computer monitor 18 is connected to the processor 16 
10 via a signal path 20. The computer monitor 18 includes a 

keyboard 22. a printer 24 is connected to the processor 16 
via a signal path 26. The processor 16 is adapted to output 
xmages in a printer readable format over the signal path 26 
and the printer 24 is adapted to output the images received 
from the processor 16 in a human perceivable printed format. 

A display monitor 28 also is connected to the processor 
16 for receiving an image from the processor 16 via a signal 
path 30 and for displaying the received image. 

The image producing means 12 , more particularly 
comprises a video camera 32. The video camera 32 responds to 
the intensity of light reflected off an image. The light 
intensity is converted into an analog signal by the video 
camera 32. The brighter the light intensity, the higher the 
amplitude of the analog signal. The video camera 32 produces 
the video images in the analog format and outputs the video 
images in the analog format over a signal path 34. 

The video images in the analog format are inputted into 
a V1 deo cassette recorder 36. The video cassette recorder 36 
records on a video medium the video images in the analog 
format received from the video camera 32 on a video tape. 

The video images in an analog format are outputted by the 
video cassette recorder 32 over a signal path 38. The video 
images outputted by the video cassette recorder 36 are 
inputted into a frame grabber 40. The video images outputted 
by the video camera 32 and the" video cassette recorder 36 
consist of a number of individual frames. The frame grabber 
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40 includes a high speed analog to digital converter and a 
high speed digital to analog converter* The frame grabber 40 
is constructed to capture a single frame of the video images 
outputted by the video cassette recorder 3 6 in the analog 
5 format and to digiti2e each scan line in the captured frame to 

produce a digitized image which is outputted over the signal 
path 14. In the frame grabber 40 each scan line is sampled at 
a constant rate. The amplitude of the analog signal which 
corresponds to the light intensity at that spot of the image 
10 now is represented by a digital number. The more light 

intensity there is at a given spot of the image, the higher 
the digital number. Each spot is called a pixel (image 
element) and the corresponding digital number is the value of 
that pixel. 

15 In the alternative, the video images in the analog format 

are outputted via the video camera 32 over the signal path 34 
and over a signal path 42 whereby the video images in the 
analog format are directly inputted into the frame grabber 40. 
In this last mentioned instance, the video cassette recorder 

20 3 6 is by-passed and a video tape record is not necessarily 

maintained of the video images outputted by the video camera 
32. In either instance, the frame grabber 40 digitizes the 
video images in the analog format and outputs the digitized 
images over the signal path 14. 

25 In the alternative, the body part to be analyzed is 

recorded via a camera 44 which is adapted to produce a still 
image or frame in a digitized format of the photographed body 
part and to output the still frame in a digital format, a 
digitized image. Thus, the camera 44 outputs the digitized 

3 0 image over. a signal path 46. 

The processor 16 is connected a disk storage 48. The 
digitized images outputted by the camera 44 over the signal 
path 4 6 or the digitized images outputted by the frame grabber 
4 0 over the signal path 14 can be inputted into the disk 

35 storage 48 over a signal path 50 and stored therein as 

directed by the processor 16. 
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in general, an individual is placed in front of the video 
camera 32 and the video camera 32 is focused on the body part 
to be analyzed. The body part to be analyzed is positioned in 

^IT 1106 ° r C ° ntr01 P ° sition - individual then moves 

the body part to be analyzed to a changed position. It should 
be noted that it may be desirable to move the body part to be 

ITlT^T" ^ contro1 position to chan * ed *° sit -n «- 

back to the control position in some applications. Also, it 
may be desirable to have the individual repeat the movement a 
number of times so the operator will have a number of images 
from which to select the desired images for processing. 

The video camera 32 produces the video images in the 
analog format of the body part to be analyzed through the 
movement of the body part from the reference position to the 
changed position. These video images outputted by the camera 
32 are recorded on video tape by the recorder 36 or directly 
xnputted into the frame grabber 40. if recorded on tape by 
the vxdeo cassette recorder 36, the recorded video images then 
are played back and outputted by the video cassette recorder 
36 and inputted into the frame grabber 40. 

The video images in the analog format are outputted from 
the frame grabber 40 via the signal path 30 to the video 
monxtor 28. Each frame of the video images also are digitized 
to produce a series of digitized images and the digitized 
xmages are outputted from the frame grabber 40 over the signal 
paths 14 and 50 and inputted in disk storage 48, as directed 
toy the processor 16. 

When it is desired to analyze the images, the digitized 
xmages are outputted from disk storage 48 over the signal path 
50 and inputted into the frame grabber 40, as directed by the 
processor 16. The frame grabber 40 converts the digitized 
xmages to video images in the analog format. The video images 
xn the analog format are outputted over the signal path 30 and 
xnputted into the video monitor 28 where the video images are 
dxsplayed. More particularly, the frame grabber 40 and the 
processor 16 are adapted to sequence through the digitized 
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omages frame by frame and each frame sequentially is displays 
on the video monitor 28. The operator using the keyboard 2 - 
instructs the processor 16 to cause the frames sequentially tc 
be displayed. 

in this capture mode of operation, the operator car 
select either one of two modes of operation: a select pair 
mode or a select sequence mode, m the select pair mode, the 
operator sequences through the images, while viewing each 
image on the video monitor 28, until the operator is satisfied 
that a particular frame represents a good image of the body 
part in the control position. The operator then operates the 
keyboard 22 to instruct the processor 16 to output a capture 
code over the signal path 14 which is inputted into the frame 
grabber 40. m response to receiving the capture code, the 
frame grabber 40 captures the frame and this captured frame in 
a digitized format is outputted by the frame grabber 4 0 is 
marked and stored in the t rocessor 16 memory or inputted into 
the disk storage 48 where it is identified as a digitized 
control frame or digitized control image. 

In the select pair mode, the operator then continues to 
sequence through the frames until the operator is satisfied 
that a particular frame represents a good image of the body 
part in the changed position. The operator then causes the 
capture code to be outputted by the processor 16. In 
response, the frame grabber 40 outputs the captured frame in 
the digitized frame which is stored in the processor 16 memory 
or inputted into the disk storage 48 where it is identified as 
a digitized moved frame or digitized changed image. 

The operator then instructs the processor 16 by way of 
the keyboard 22 to process the selected control frame and the 
selected changed frame. In the process mode of operation, the 
processor 16 causes the digitized control image and the 
digitized changed image to be withdrawn from disk storage 48 
into the processor 16 memory if they are not already 
previously stored in the processor 16 memory. The processor 
16 then compares the digitized control image to the digitized 
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changed image and the processor 16 produces a digitized 
compare frame or digitized compare image which is indicative 
of the movement of the body part from the control frame or 
image to the moved or changed frame of image. The processor 
5 16 is programmed to determine movement parameters indicative 

of the movement from the control image relative to the changed 
frame or image and to output the movement parameters, as well 
as to cause the digitized compare image to be processed 
through frame grabber 40 which outputs the compare image in 
10 the analog format. The compare image in the analog format is 

outputted over the signal path 30 for display on the video 
monitor 28. 

In addition, the determined movement parameters are 
outputted over the signal paths 14 and 20. The determined 
15 movement parameters are displayed on the computer monitor 18 

screen. The processor 16 also can cause the determined 
movement parameters, as well as the compare image, to be 
outputted on the signal path 26 for outputting the compare 
image and the determined movement parameters in the printed 
20 format via the printer 24. 

As mentioned before, the operator also can select the 
select sequence mode of operation, rather than the select pair 
mode of operation just described. In the select sequence mode 
of operation, the video images are produced and each frame of 
25 the video images is digitized via the frame grabber 40 and 

stored in the processor 16 memory or disk storage 48 in a 
manner exactly like that described before with respect to the 
select pair mode of operation - 

In the select sequence mode of operation, the operator 
3 0 sequences through the images while viewing each image on the 

video monitor 28- Using the keyboard 22, the operator selects 
a particular frame which represents a good image of the body 
part in the control position and the operator instructs the 
processor 16 to mark the selected frame. 
3 5 After selecting the control frame, the operator then 

sequences through the frames and selects a plurality of moved 
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or changed frames. For example, the operator may select a 
plurality of moved or changed frames between the control frame 
showing the body part in the control position through the 
frames showing the body part in the changed position, and back 
5 through a frame showing the body part in a position similar to 

the control position. 

The control frame and the moved or changed frames each 
are assigned a number. In one present embodiment of the 
invention, up to thirty-six frames can be selected as the 
10 control frame and the moved or changed frames. The number of 

frames that can be selected is limited only by the 
availability of the processor 16 memory. The selected control 
rame and each of the selected moved or changed frames are 
stored in the disk storage 48 along with a reference number 
15 identifying each of the selected frames. 

The operator then instructs the processor 16 by way of 
the keyboard 22 to process the selected frames. The processor 
16 is conditioned in the process mode of operation in response 
to this command. 

20 In the process mode of operation, processor 16 causes the 

digitized control frame and the next selected moved frame to 
be withdrawn from disk storage 48 into the processor 16 memory 
if they are not previously stored there already. The 
processor 16 then compares the digitized control image or 

25 frame to the digitized changed image or frame and the 

processor 16 produces a first digitized compare frame or first 
digitized compare image which is indicative of movement of the 
body part from the control frame or image to the moved or 
changed frame or image. The processor 16 also is programmed 

30 to determine movement parameters indicative of the movement 

change from the control frame or image relative to the moved 
or changed frame or image. 

The processor 16 then compares in sequence each of the 
moved or changed frames or images with the control frame or 

35 image, to produce a plurality of digitized compare frames or 

images, each of which is indicative of the movement of the 
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body part between the control frame or image and each 
respective moved or changed frame. The processor 16 is 
programmed to determined movement parameters indicative of the 
movement change from the control frame or image relative to 
each of the moved frames or images. The movement parameters 
determined by the processor 16 are outputted by the processor 
16 over the signal path 20 and displayed on the screen of the 
computer monitor 18. m addition, the movement parameters 
xndicative of movement change can be outputted over the signal 
path 26 and printed by the printer 24 to provide a list of the 
determined parameters in a printed format. 

Video cameras such as the video camera 32, video cassette 
recorders such as the video cassette recorder 3 6 and cameras 
such as the camera 44 are all well known in the art and a 
detailed description of the construction and operation of 
these components is not deemed necessary. The format of the 
video tape used in the video cassette recorder 3 6 can be Beta, 
VHS, PAL, or 1/2" or 3/4" video tape or any other suitable 
video tape format. Also it should be noted that a video disk 
recorder can be used in lieu of the video cassette recorder. 

Frame grabbers which are constructed to operate in a 
manner described herein with respect to the frame grabber 40 
are well known in the art and a detailed description of the 
construction and operation of such frame grabbers is not 
25 deemed necessary. For example, one frame grabber suitable for 

use with the present invention is a frame grabber sold by Data 
Translation, and identified by Model DT-2853. This frame 
grabber is constructed to receive video images and to receive 
capture commands and to isolate and capture a selected frame 
3 0 in the video images received by the frame grabber as 

instructed by the received capture command. This commercially 
available frame grabber also is adapted to digitize each scan 
line in the selected or capture frame. This type of frame 
grabber also is constructed to receive digitized images and 
convert such digitized images to their analog formats for 
viewing in human perceivable forms such as via the display 
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monitor 28. 

A processor such as the general purpose digital computer 
available from Master Systems, Model Executive AT has been 
found suitable for use as the processor 16 of the present 
5 invention. This processor was used in connection with a disk 

storage commercially available Seagate, Model 225, and this 
svstem was found suitable for use in the present invention. 

It should be noted that there are available other types 
of frame grabbers which can be used to combine the digital 

10 image with messages from the processor 16 using a digital 

video interactive (DVI) protocol. In this case, only one high 
resolution video monitor is required. Further, in lieu of the. 
keyboard 22 used with the computer monitor 18, an optional 
interface box with push buttons can be used to communicate 

15 with the processor 16. 

To detect motion, the subregions of the same image that 
have been moved need to be identified. This requires at least 
two images. The first one is referred to herein as the 
control image or control frame. The second image is the 

20 changed image where subregions of the body part have been 

relocated referred to herein as the moved image or moved 
frame. In the case of facial motion analysis, the control 
image can be the face at rest while the second or moved image 
can be the face after a specific feature, such as the eyebrow, 

25 has been moved. Alternatively, the comparison also can be 

made between the left half and the right half of the same 
image or frame. 

In the processing mode of operation, the processor 16 
performs four distinct processes: (l) image pre-processing? 

30 (2) motion detection (3) image post processing and (4) 

measurement . 

IMAGE PRE-PROCESSING 

In the image pre-processing step, digitized images are 
35 processed to yield a consistent image quality. This is an 

optional step. Due to differences in lighting between images 
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and between the various features of the image, pre-processing 
is necessary to equalize or normalize the pixel ranges of the 
digitized images or frames. The lighting can be changed from 
one image to another at the same photo session or between 
5 series of images. The equalization process compensates for 

the varying intensity and minimizes the shadow zones often 
occurring from three dimensional objects. in the system of 
the present invention, the equalization process is 
accomplished by equalizing the histogram of pixel values into 
10 a specific distribution. This equalization step produces 

images that have an average intensity throughout and with a 
specific distribution of pixel intensity variations. 

The captured images are usually taken under varying 
lighting conditions dependent on the environmental setting. 
Even when the setting is carefully controlled, the light 
reflected from the face also may vary according to the 
complexion, skin color, head shape, skin texture and other 
facial features. Therefore, it is essential to normalize the 
intensity of the image. From the existing intensity 
histogram, the pixel levels are remapped to give a specific 
histogram shape such as a bell curve, a Poisson curve, a 
linear curve, a logarithmic curve, or a user defined curve. 
In the present invention, a bell curve was utilized so that 
the equalized images had the same average intensity and the 
same intensity deviation for later comparisons between two 
images or between sets of images. 

MOTION DETECTION 

After pre-processing the images or frames, the subregions 
of the image or frame that are different between images are 
highlighted. This is done by comparing two images or frames 
and removing areas that are common in the two images or 
frames. when the common or subregions are suppressed, only 
the regions that have been moved remain. This is the main 
operation for detecting motion. By examining the two images, 
the common area can be subtracted out thus leaving a small 
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residual pixel value. In an idea case, if the two images were 
identical, there would be no residual value for all pixels. 
If any subregion in the image have been relocated, a 
substantial difference in pixel values results after the 
5 subtraction. The relocated region can now readily be 

differentiated by their pixel values from the other common 
regions . 

After the subtraction of the two images, negative values 
may arise. Three methods are possible for handling such 

10 negative values: (1) negative values are treated as zero; (2) 

negative values are changed to absolute values; and (3) all 
values are scaled and offset to eliminate negative values. 
When negative values are treated as zero, most moved regions 
are highlighted. When negative values are changed to absolute 

15 values, a true difference between two images or frames is 

obtained. This last mentioned scheme may provide more 
accurate assessment of the moved regions • When all values are 
scaled and offset to eliminate the negative values, the 
difference in images or frames seen may be advantageous for 

20 some type of measurement. In the system of the present 

invention, negative values were changed to absolute values. 

IMAGE POST PROCESSING 

After the common areas between images have been removed, 

25 the last step is image post processing. This is an optional 

step . In this step , an image enhancement process first is 
utilized to bring out the regions that have been mov&d. Then, 
a digital filter in software may be used to clean up unwanted 
and spurious spots in the image. 

30 For image enhancement, it is desirable, after image 

subtraction, to highlight the moved regions for visual study 
and verification. The subtracted image may be enhanced 
through a number of techniques: (1) contrast enhancement, or 
f 2) thresholding. In contrast enhancement, the subtracted 

35 sage is enhanced so that the brighter regions appear to be 

more bright. This essentially is a histogram equalization 
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process. In thresholding, the gray scale is divided so that 
the pixels above a certain threshold appear extremely bright, 
while those pixels below the threshold appear extremely dark. 
In other words, the image is thresholded so that a small 
5 residual errors in the common regions are replaced by a null 

value, while those pixels representing the dislocated or moved 
regions that are above the threshold are amplified to the 
maximum values. The common regions have zero pixel values and 
the dislocated or moved regions have maximum pixel values. 

10 The threshold can be set manually or automatically. In one 

embodiment of the present invention, thresholding was utilized 
in the image post processing step and the threshold values 
were set manually by the operator using the keyboard 22. 

It is not unusual that, after the image subtraction 

15 process and the image enhancement process, many spurious 

points may remain in the image. The spurious points are 
usually isolated points placed randomly around the image. 
Therefore, a further step in post processing is to remove 
these isolated, spurious points. This may be done by a 

2 0 cleansing filter such as a Sobel filter that sharpens the 

enhanced regions and deemphasizes the isolated points. At 
this stage of the process, the remaining image or frame is 
ready for quantitative measurements. 

25 MEASUREMENT 

The final step is to quantify (determine the movement 
parameters) the regions of interest and to measure the 
movement in terms of the brightness of the subtracted image. 
This quantification step can be accomplished in a number of 

3 0 different ways. Two possible approaches are pixel counting 

and area integration. In pixel counting, the area of movement 
is measured by the number of pixel above a certain threshold. 
Therefore, the moved region is approximated by counting the 
number of pixels above the threshold. In area integration, 
35 the moved region is approximated by the summation of the 

actual subtracted values, i.e., the difference of the two 



WO 92/03801 PCT/US91 /05980 



15 

images in gray level. The last mentioned approach may be more 
indicative of the movement and possibly the effort of 
movement. In one operational embodiment of the present 
invention, pixel counting was used to quantify the regions of 
5 interest and to measure the movement in terms of the 

brightness of the subtracted image. 

shown in Figure 2 is a diagrammatic view of the software 
or processing operations performed via the processor 16. In 
other words, the processor 16 is programmed to operate in this 
10 manner. 

The program in the processor 16 first is invoked and 
initialized (START 52 in Figure 2). In response to receiving 
a capture code inputted into the processor 16 via the computer 
keyboard 22 by the operator, the processor 16 is conditioned 
15 in the capture mode of operation (Image Capture 54 in Figure 

2). 

In the capture mode of operation, the digitized images 
are captured and transferred to the processor memory and 
optionally the disk storage 48. 

20 After the digitized images have been captured, the 

captured images can be previewed on the display monitor 28 
(Image Preview 56 in Figure 2) to insure that the digitized 
images of interest have in fact been captured and stored. 

After previewing the digitized images, the operator can 

25 select the particular images to be stored for later selection 

and processing (Image Selection 58 in Figure 2) . It should be 
noted that during the image preview 56 or the image selection 
58, the digitized images are reconverted into images visually 
perceivable in a human viewable format and outputted on the 

30 video monitor 28. The operator or user then uses keyboard 

22 of the computer monitor 18 to effect the . action of 
images being displayed on the display monitor 28 . The 
selected digitized images are stored in the disk storage 48 as 
controlled by the processor 16 (Stored Image 60 in Figure 2) 

35 or the digitized images may be inputted directly from the 

camera 44 (Digital Still 62 in Figure 2). 
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In any event, the selected digitized control image and 
the selected digitized moved or changed image are taken either 
from the disk storage 48 where the images previously have been 
stored or from the frame grabber 40 where the images were just 
5 captured or from the digitized still camera 44 . The images to 

be analyzed by the processor 16 are selected by the user and 
the user marks the areas of interest in the selected digitized 
control image and the selected digitized changed image. The 
digitized control image and the digitized changed image then 

10 are processed via the processor 16, 

In the process mode of the processor 16, the selected 
images (a selected digitized control image and the selected 
digitized changed images) are screened and analyzed to remove 
various artifacts. The purpose of this step is to eliminate 

15 white spots and to account for any differences in skin 

pigmentation, uneven lighting reflection and undesirable image 
translation. The primary tool used to accomplish this step is 
histogram equalization indicated in Figure 2 by the reference 
numeral 64 . 

20 After histogram equalization 64, the digitized control 

image and the digitized changed image are compared through a 
subtraction process (Subtraction 66 in Figure 2) or a 
difference process (Different 68) or a biased subtraction 
process (Bias Subtraction 70) to produce a digitized compare 

25 image. The digitized compare image then is processed to 

remove artifacts and the moved regions in the digitized 
compare image are enhanced. This step is accomplished by way 
of histogram equalization 72. 

After histogram equalization 72, the digitized compare 

3 0 image is processed to quantify the movement either through the 

counting of threshold pixels (Threshold 74 in Figure 2) or 
through integration of pixel gray values in the area of 
interest (Gray Value 76 in Figure 2) . After the measurement 
step, the processor 16 software returns to the main menu for 

35 further processing (END 75 in Figure 2) . 

It should be noted that at any step in the image capture 
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or processing or detection or post processing or measurement 
steps, the operator has the option to return to the main menu. 

A software listing which can be used to operate processor 
16 in one operational embodiment of the invention can be found 
5 in U.S. Patent Application Serial No. 573,502, filed August 

24, 1990. With that software, a logo screen initially is 
invoiced and then a main menu is presented. The main menu in 
this case has four primary modes: the capture mode; the load 
and store mode; the process mode and the configuration. When 

10 the capture mode is selected, another screen is presented that 

solicits from the user whether or not the user wants to 
proceed or change parameters. If the change parameter submode 
is selected, then the system parameters related to image 
capture are made available for the user to change and modify. 

15 Some typical parameters at this point are normal frame rate, 

slow frame rate, number of frames for example. If the user 
selects to proceed with the capture mode, another screen 
called the capture selection screen is presented where the 
user can specify the normal rate or slow rate. After the 

20 selection of capture rate, another screen is presented to the 

user where the capture process can be initiated. After the 
capture process is completed, the user can select to view the 
captured images individually or as a whole on the display 
monitor 28 . If the user chooses the load and store mode , 

25 several options are available. These options included 

retrieving from the disk storage 48, saving to the disk 
storage 48 and viewing the current images individually or as 
a whole on the display monitor 28. After the images are 
viewed via the display monitor 28, the user then is asked to 

3 0 select frames for processing. The user can pass through the 

frames using cursor keys and marking the frames designated for 
later analysis. The frames can be marked in two ways. First, 
pairs can be selected individually for analysis or, 
alternately, a series of frames can be selected for comparison 

35 with a particular reference frame. After the frames are 

selected, the system automatically returns to the main menu 
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where -the user can select the process mode. 

In the process mode, screens are presented to the user as 
an example of the procedure. The processing of the images 
begins with a normalization procedure of the control image, 
5 then the subtraction step between the control image and the 

changed image is effected. At this point, the user is asked 
to define the coordinates of the body part or face. This is 
done without the use of marker dots or with the use of marker 
dots on the face or marker dots on the head holder if 
10 available. This step may be done automatically if maker dots 

are present. After the coordinates are defined, then the user 
is asked to define the areas of interest using cursor keys. 
After the areas of interest are defined, the system then 
proceeds to measure new regions within the defined areas of 
15 interest. For random pairs, the system proceeds in the same 

manner, after the definition of coordinates in the boxes, the 
system can enter a batch mode and process all of the images at 
the time. The output of the measurement is recorded on a file 
for storage or for later printout. After processing, the 
20 system return to the main menu for further instruction. 

As mentioned before, the present invention particularly 
is adapted to detect and analyze facial movement. 

Diagrammatically shown in Figure 3 is a patient 80 or, 
more particularly, a patient's face. Initially, marker dots 
82, 84, 86 and 88 are placed on the patient's face, as shown 
in Figures 4 and 5. The mark dots 82, 84, 86 and 88 provide 
a means to align images or frames. Also, shown in Figures 5, 
6 and 7 is a head holder 90, and the marker dots or at least 
three marker dots could be placed on portions of the head 
holder 90, rather than on the patient's face. 

It should be noted that other means also could be 
utilized to align images or frames for processing. Also, it 
should be noted that, in one embodiment of the invention, 
marker dots were not utilized. if marker dots are not 
35 utilized, it is important that the patient's 80 head be 

secured in the same position between images and the camera 32 
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be positioned in exactly the same position between images with 
exactly the same camera 32 settings. The head holder 90 
comprises a holder stand 92 with only the upper portion of the 
holder stand 92 being shown in Figures 6 and 7. The holder 
5 stand 92 may be adapted to be supported on a floor surface or, 

in the alternative, the holder stand 92 may be adapted to be 
mounted or supported on a wall surface. The holder stand 92 
preferably is cylindrically shaped, although the holder stand 
92 could have any other shape if desired in a particular 
10 application. 

The head holder 90 includes a top pad assembly 94 and a 
chin pad assembly 96. The top pad assembly 94 is movably 
connected to the holder stand 92 for movement in a vertically 
upwardly direction 98 (Figure 6) and in a vertically 
15 downwardly direction 100 (Figure 6) . 

The top pad assembly 94 comprises a top bar 102, a top 
sleeve 104 and a pad 105. The top sleeve 104 has an opening 
(not shown) extending therethrough and the holder stand 92 is 
slidingly disposed through the opening (not shown) in the top 
2 0 sleeve 104. One end of the top bar 102 is connected to the 

top sleeve 104 and the pad 105 is connected to the opposite 
end of the top bar 102. 

The chin pad assembly 96 comprises a U-shaped chin bar 
106, a chin sleeve 108 and a pair of chin pads 110 and 112. 
25 The chin sleeve 104 has an opening (not shown) extending 

therethrough and the holder stand 92 is slidingly disposed 
through the opening (not shown) in the chin sleeve 108. A 
mid-portion of the chin bar 106 is connected to the chin 
sleeve 108. One of the chin pads 110 is connected to one end 
of the chin bar 106 and the other chin pad 112 is connected to 
the opposite end of the chin bar 106. 

In operation, the patient 80 is seated in front of the 
head holder 90 with the patient's 80 head disposed generally 
between the top pad assembly 94 and the chin paid assembly 96. 
35 The top pad assembly 94 is moved in the downwardly direction 
100 until the pad 105 engages a central portion of the top of 
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the patient's 80 head. The top pad assembly 94 is secured in 
this position via a fastener X14 which extends through the top 
sleeve 104 and engages the holder stand 92 to secure the top 
sleeve 104 in the one adjusted position on the holder stand 
5 92. Then, the chin pad assembly 96 is moved in the upwardly 

direction 98 to a position wherein the pad 110 engages one 
side of the patient's 80 chin and the pad 112 engages the 
opposite side of the patient's 80 chin. The chin pad assembly 
9 6 is secured in this position via a fastener 116 which 

10 extends through the chin sleeve 108 and engages the holder 

stand 92 to secure the chin sleeve 108 in the one adjusted 
position on the holder stand 92. The patient's 80 head is 
held in this stationary position secured between the top pad 
assembly 94 and the chin pad assembly 96. 

15 The video camera 32 is positioned in front of the patient 

80. The video camera 32 is focused on the patient's 80 face 
with the patient's 80 face occupying virtually or 
substantially the entire image seen by the video camera 32. 

It is important to adjust or create the proper lighting 

20 about the patient's 80 face. In one embodiment of the present 

invention, it has been found that soft, low power lighting 
which comes in from both sides of the patient's 80 face, 
rather than harsh, studio bright lighting, yields the best 
results . 

25 With the patient's 80 head secured in the head holder 90, 

the video camera 32, the video cassette recorder 36 and the' 
video monitor 28 are turned on. The video cassette recorder 
36 is conditioned in the record mode and the video camera 32 
also is conditioned in the record mode. The video images in 

3 0 the analog format outputted by the video camera 32 over the 

signal path 34 are inputted into the video cassette recorder 
36 which records the video images on video tape. In addition, 
the video images are outputted over the signal paths 42 and 38 
through the frame grabber 40 which outputs the video images 

35 over the signal path 30. The video images outputted over the 

signal path 30 are inputted in the video monitor 28 which 
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displays the received video images. In this manner, the 
operator is able to view the images being recorded via the 
video cassette recorder 36 to be sure that proper or desired 
images in fact are being recorded. 
5 When using the motion analysis apparatus 10 for analyzing 

facial movements, the operator, for example, would request the 
patient SO to perform four tasks: (1) raise eyebrows, (2) 
close eyes tightly, (3) squint or wrinkle nose, and (4) smile 
maximally. Each of these tasks is performed by the patient 80 
10 three times to be sure a proper and usable sequence is 

recorded. In addition, each task begins with the patient's 80 
face at rest, then moved to maximum movement and then returned 
to rest. 

After the recording of the video images, the video 
15 cassette recorder 36 is conditioned in the playback mode. The 

video images are outputted by the video cassette recorder 36 
over the signal path 38 and the video images are inputted into 
the frame grabber 40. The video images are passed through the 
frame grabber 40 and over the signal path 3 0 and inputted into 

2 0 the video monitor 28 where the video images are displayed. 

The video images inputted into the frame grabber 40 are 
digitized and the frame grabber 40 outputs the digitized 
images or frames over the signal path 14. The digitized 
images are inputted into the disk storage 48. 
25 The operator then conditions the processor 16 in the 

capture mode. In the capture mode, the operator can 
sequentially output the digitized images or frames from the 
disk storage 48 and input such digitized images or frames into 
the frume grabber 40. The frame grabber 40 converts each 

3 0 digitized image into an image in the analog format and the 

image in the analog format is outputted over the signal path 
30 for display on the video monitor 28. 

At this stage, the operator can select a select pair mode or 
a select sequence mode of operation by inputting to the 
35 processor 16 via the computer keyboard 22 a select pair 

command or a select sequence code. 
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Assuming the operator has selected the select pair mode 
and again referring to the facial movement example, the 
operator sequences through the frames. In this instance, the 
operator selects a control frame where the patient's 80 face 
is at rest and a moved frame where the patient's face has 
experienced maximum movement such as where the patient 80 has 
raised the patient's 80 eyebrow to maximum movement. The 
frames are selected by the operator inputting a capture code 
via the computer keyboard 22 to the processor 16. The 
selected frames are outputted by the frame grabber 40 over the 
signal paths 14 and 50 and stored in the disk storage. 

After the control frame and the moved frame have been 
captured, the operator then inputs a process code to the 
processor 16 via the computer keyboard 22 to condition the 
processor 16 in the process mode. In the process mode, the 
processor 16 takes the control frame and the moved frame from 
the disk storage 48 and the processor 16 compares the control 
frame to the moved frame to produce a compared frame. 

The processor 16 then produces a gray frame in a digital 
format. The gray image is produced from the compared frame by 
taking only the pixels having a light value above a 
predetermined light value, for example, above 22. The 
digitized gray frame is converted to a gray frame in an analog 
format by the frame grabber 40 and outputted over the signal 
25 path 30 and inputted into the display monitor 28 which 

displays the inputted gray frame. The gray frame provides 
somewhat of an outline of the patient's 80 head and facial 
features for the purpose of selecting areas of interest, in a 
manner which will be described in greater detail below. 
30 After the gray frame has been displayed on the screen of 

the display monitor 28, the operator selects a center of the 
displayed image. In the example of analyzing facial motion, 
a center generally is picked about the end of a patient's 80 
nose, a center lis being shown in Figure 3 for example. For 
35 the purpose of illustrating the selection of areas of 

interest, it is assumed that Figure 3 represents the gray 
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image displayed on the screen of the display monitor 28. 

After the selection of center 118, a horizontal grid 12 0 
and a vertical grid 122 are displayed on the screen of the 
display monitor 28 with the grids 120 and 122 intersecting at 
the selected center 118. Using the computer keyboard 22, the 
processor 16 is programmed to permit the operator to select an 
area of interest. Assuming the motion being analyzed is the 
raising of the patient's 80 eyebrow, the area of interest 
selected would be the area of interest 124 indicated by dashed 
lines in Figure 3 which includes the area of the face which 
should have been moved or caused to be moved by the raising of 
the patient's eyebrow. In one embodiment, the operator 
selects an area of interest 126 on the right side of the 
patient's face. The processor 16 automatically selects and 
15 outlines a corresponding area of interest 128 on the left side 

of the patient's 80 face. After the area of interest 124 has 
been selected, the operator then selects either the area of 
interest 126 or the area of interest 128 to be designated as 
the control position and the opposite area of interest 126 or 
20 128 to be the moved position. 

The operator then selects a threshold. The threshold is 
a value of the light intensity of a pixel and only pixels 
having a light value above the threshold are selected for 
analyzing. In the present embodiment of the invention, the 
25 operator has a number of thresholds which the operator can 

select depending on the desires of the operator and depending 
upon which threshold level gives the best comparison in the 
opinion of the operator. For example, a selected threshold 
might be 22. 

30 After the threshold has been selected, the pr sssor 16 

then counts the number of pixels in the selected areas 126 and 
128 having a light valve above the selected threshold, and 
determines the movement change using the counted pixels above 
the selected threshold in the area of interest 124. The 

35 processor 16 also determines a percent of movement between the 

left area of interest 126 and the right area of interest 128. 
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For example, in one patient using one embodiment of the 
present invention with the selected areas being about the same 
as the selected areas 124, 126 and 128 and with the patient's 
80 requested movement being the raising of the patient's 80 
eyebrows, the following movement parameters were determined by 
the processor 16 and displayed on the screen of the computer 
monitor 18: 

TABLE I 



10 TOP LEFT QUADRANT : 71 

DIFF: 101 %(L/R): 41.28 

TOP RIGHT QUADRANT : 172 



%(R/L): 242.25 



BOTTOM LEFT QUADRANT : 0 

15 DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



% (R/L) 



0.00 



20 



25 



30 



35 



The numbers associated with the DIFF parameter in Table 
I above are arbitrary numbers representing the change in 
movement in the areas of interest 12 6 and 128 between the 
control frame and the changed frame. The TOP LEFT QUADRANT 
TOP RIGHT QUADRANT, BOTTOM LEFT QUADRANT, and BOTTOM RIGHT 
QUADRANT listed in Table I are pixel counts that represent the 
number of pixel in the areas of interest 126 and 128 
respectively having a light value above the selected threshold 
in the compare frame. The percent movement numbers in Table 
1 above represent a ratio of the pixel counts between the left 
and the right side. 

Assuming that the operator desires to select a sequence 
of frames to be analyzed, the process is identical to the 
process described before with respect to selecting pairs for 
analyzing, except instead of initiating a select pair code, 
the operator initiates via the computer keyboard 22 a select 
sequence code which is inputted into the processor 16 over the 
signal path 20. m the select sequence mode, the operator 
selects a sequence of frames to be analyzed. m one present 
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embodiment of the invention, an operator can select up to a 
maximum of thirty-six frames for analyzing in the select 
sequence mode of operation. 

For example and assuming that the requested movement 
again was the raising of the patient • s 80 eyebrow , the 
operator might select a sequence of thirty-six frames starting 
with the patient's face in a rest position and then 
sequentially selecting a number of frames between the 
patient's face in the rest position through the maximum 
movement where the patient's eyebrows are raised and back to 
the rest position. Each of these frames is captured in the 
manner described before. 

After the sequence of frames has been captured, the 
operator then selects a reference pair of frames from the 
selected frames, such as the first frame where the patient* s 
face is at rest and one of the frames showing maximum movement 
of the patient's face where the eyebrow is in the raised 
position. The processor 16 compares the reference pair of 
frames and produces the gray frame in the manner described 
before with respect to the select pair mode of operation. 
After the gray frame has been produced, the gray frame is 
displayed on the computer monitor 18 and the areas of interest 
are selected in the manner described before with respect to 
the select pair mode of operation. 

After the areas of interest have been selected, the 
operator then selects a control frame such as the first frame 
showing the patient's face in the control or rest position. 
The remaining frames in the sequence of frames are designated 
moved frames. The processor 16 then compares each moved frame 
with the control frame in the selected pairs sequentially. 
The reference frame and the moved frames are compared in 
sequence in a manner exactly like that described before with 
respect to the select pair mode of operation to produce a 
plurality of compare frames- After the processor 16 has 
compared sequentially the moved frames with the, control frame 
in sequence, the processor 16 then outputs the comparison 
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information similar to Table I, except the information would 
be included for the comparison of each of the moved frames 
with respect to the control frame in the selected sequence. 
For example, Table II below shows the output of the processor 
16 where the requested movement was the raising of the 
patient's 80 eyebrow and the selected frames included the 
patient's 80 face at rest, through the movement of the 
eyebrows to a maximum moved position, and back to the 
patient's 80 face at rest. 

TABLE II 



PAIRS 10-11 



PATIENT NAME 



CARD NO. : 



15 TOP IjEFT QUADRANT : 71 

DIFF: 101 %(L/R): 41.28 

TOP RIGHT QUADRANT : 172 



%(R/L): 242.25 



BOTTOM LEFT QUADRANT : 0 

20 DIFF: 0 %(L/R): 0-00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



25 



PAIRS 10-12 



PATIENT NAME: 



TOP LEFT QUADRANT : 0 

DIFF: 40 %(L/R): 0.00 

TOP RIGHT QUADRANT : 40 



CARD NO. : 



%(R/L) 



3 0 BOTTOM LEFT QUADRANT : 0 

DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT ; 0 



%(R/L) : 0.00 
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PATIENT NAME 



CARD NO. : 



TOP LEFT QUADRANT : 3085 

DIFF:-1823 %(L/R): 244.45 

TOP RIGHT QUADRANT : 1262 



%(R/L): 40-91 



10 



BOTTOM LEFT QUADRANT * O 

DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



PAIRS 10-14 



PATIENT NAME: 



CARD NO 



TOP LEFT QUADRANT : 5758 

15 DIFFi-3202 %(L/R): 225.19 

TOP RIGHT QUADRANT : 2557 



%(R/L) : 44.41 



20 



BOTTOM LEFT QUADRANT : 0 

DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



PAIRS 10-15 



PATIENT NAME: 



CARD NO. : 



25 TOP LEFT QUADRANT : 5841 

DIFF:-3099 %(L/R): 213.02 

TOP RIGHT QUADRANT : 2742 



%(R/L): 46.94 



BOTTOM LEFT QUADRANT : 0 

30 DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 
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28 

PATIENT NAME 



TOP LEFT QUADRANT : 59 18 

DIFF:-2988 % (L/R) : 201.98 

TOP RIGHT QUADRANT : 2930 

BOTTOM LEFT QUADRANT : 0 

DIFF: 0 %(L/R): o.OO 

BOTTOM RIGHT QUADRANT : 0 



CARD NO 



%(R/L): 49.51 



%(R/L): 0.00 



10 



15 



20 



PAIRS 10-17 



PATIENT NAME: 



TOP LEFT QUADRANT : 6098 

DIFF: -3284 %(L/R): 216.70 

TOP RIGHT QUADRANT : 2814 

BOTTOM LEFT QUADRANT ; 0 

DIFF: 0 %(L/R): o.OO 

BOTTOM RIGHT QUADRANT : 0 



CARD NO. : 



%(R/L): 46.15 



%(R/L): 0.00 



25 



30 



PAIRS 10-18 



PATIENT NAME: 



TOP LEFT QUADRANT : 6420 

DIFF: -3397 %(L/R): 212.37 

TOP RIGHT QUADRANT :3023 

BOTTOM LEFT QUADRANT ; 0 

DIFF: O %(L/R): 0 .00 

BOTTOM RIGHT QUADRANT- : 0 



CARD NO. 



%(R/L): 47.09 



%(R/L): 0.00 
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29 

PATIENT NAME 



CARD NO . : 



TOP LEFT QUADRANT : 6315 

DIFF:-2862 %(L/R): 182.88 

TOP RIGHT QUADRANT : 3453 



%(R/L): 54.68 



10 



rtTTH HD1MT 



: 0 



DIFF: 0 %(L/R): °- 00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



PAIRS 10-20 



PATIENT NAME: 



CARD NO 



TOP LEFT QUADRANT : 6552 

15 DIFF:-3334 % (L/R) : 203.60 

TOP RIGHT QUADRANT : 3218 



%(R/L): 49.11 



20 



BOTTOM LEFT QUADRANT : 0 

DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



PAIRS 10-21 



PATIENT NAME: 



CARD NO. : 



25 



TOP LEFT QUADRANT : 64 06 

DIFF:-3030 %(L/R): 189.75 

TOP RIGHT QUADRANT : 3376 



%(R/L): 52.70 



BOTTOM LEFT QUADRANT : 0 

30 DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



35 
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PAIRS 10-22 



30 

PATIENT NAME 



TOP LEFT QUADRANT : 3748 

DIFF:-1493 %(L/R): 166.21 

TOP RIGHT QUADRANT : 2255 

BOTTOM LEFT QUADRANT : o 

DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



CARD NO. : 



%(R/Ii): 60.17 



%(R/L): 0.00 



10 



15 



PAIRS 10-23 



PATIENT NAME 



TOP LEFT QUADRANT : 1295 

DIFF: 131 %(L/R); 90.81 

TOP RIGHT QUADRANT : 1426 



CARD NO.; 



%(R/L) : 110.42 



20 



BOTTOM LEFT QUADRANT ; 0 

DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



PAIRS 10-24 



PATIENT NAME: 



25 TOP LEFT QUADRANT : 433 

DIFF: 646 %(L/R): 42.78 

TOP RIGHT QUADRANT : 1129 



CARD NO. 



%(R/L): 233.75 



BOTTOM LEFT QUADRANT : 0 

30 DIFF: 0 %(L/R): 0.00 

BOTTOM RIGHT QUADRANT : 0 



%(R/L): 0.00 



35 
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PAIRS 10-25 PATIENT NAME: CARD NO - : 

TOP LEFT QUADRANT : 640 

DIFF: 842 %(L/R): 43.18 %(R/L): 231.56 

TOP RIGHT QUADRANT : 1482 

BOTTOM LEFT QUADRANT : 0 

DIFF: 0 %(L/R): 0.00 % (R/L) : 0.00 

BOTTOM RIGHT QUADRANT : 0 



Table II gives a quantitative picture of the movement of 
the patient's face from a rest position, to a raised eyebrow 
position and back to a rest position. Thesholded and non- 
thresholded gray scale pixel intensity difference values are 

15 plotted against rest versus motion pairs. A pair is the 

reference control frame of the patient's face at rest compared 
to sequentially changed frames with the patient's face in a 
requested motion. Pair one, for example, would be the control 
frame subtracted from the next changed frame in sequence. 

20 Pair two, would be the control frame subtracted from the 

second changed frame in sequence. Pair three would be the 
control frame compared to the third changed frame in sequence. 

By this plotting method, a dynamic function curve can be 
plotted, resulting from a given facial motion request, such as 

25 raising the brow, closing the eyes, wrinkling the nose, or 

smiling. This dynamic function has a steep contraction phase, 
a less steeply sloped end of contraction phase, a duration of 
maximum contraction, then a steeply sloping relaxation phase. 
This function has amplitude, duration, and an inscribed area 

30 as well. 

A variety of data can be calculated from the dynamic 
curves, such as slope, intercept, duration, maximum amplitude, 
and integrated area under the curve. Each parameter of data 
then can be normalized to normal populations of varying ages 

35 to which similar data parameters from abnormal populations may 

be compared. By these methods, degrees of facial paresis and 
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abnormal facial movements, such as synkinesis, may be 
quantitatively compared to normals and to the normal side of 
a given patient's face. 

In some instances, one will analyze a left side and a 
5 right side of a patient in a single position or actually 

unchanged position. In this instance, the left side or one 
component part of the frame would be designated the reference 
position and the right side or one other component part of the 
frame would be designated the changed position, as those terms 

10 are used in the claims. 

Changes may be made in the construction and the operation 
of the various components, elements and assemblies described 
herein and changes may be made in the step or the sequence of 
steps of the methods described herein without departing from 

15 the scope of the invention as defined in the following claims. 
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Claims 



5 1. A method for detecting motion of a body part 

comprising: 

producing a digitized frame of the body part in a 
reference position of the body part, identified as a digitized 
control frame ; 

10 producing one or more digitized frames of the body 

part in a moved position of the body part identified as one or 

more digitized changed frames; and 

comparing the digitized control frame with each of 

the digitized changed frames and determining shape changes 
15 over a defined surface area of the body part from the control 

position to the moved position. 

2 . The method of claim 1 wherein the step of 
producing one or more digitized changed frames comprises: 

20 producing a plurality of digitized frames of the 

body part as the body part is moved from the reference 
position to the moved position and designating the digitized 
frames as digitized changed frames; and 

comparing each digitized changed frame with the 

25 digitized control frame to determine a parameter for each 

comparison with each movement parameter being indicative of 
motion of the body part between the digitized control frame 
and the digitized changed frame compared with the digitized 
control frame. 

30 

3 . The method of claim 1 or 2 wherein the step of 
producing the digitized control frame further comprises: 

videoing using a video camera the body part to 
produce video images of the body part in an analog format; 
35 grabbing a frame of the video images in the analog 

format showing the body part in the control position; and 
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digitizing the frame of the video image in the 
analog format and in the control position to produce the 
digitized control frame; and wherein the step of producing the 
one or more digitized changed frames further comprises: 
5 videoing using a video camera the body part to 

produce video images of the body part in an analog format; 

grabbing one or more frames of the video images in 
the analog format showing the body part in one or more changed 
positions; and 

10 digitizing the one or more frames of the video image 

in the analog format and in the one ore more changed positions 
to produce the one ore more digitized control frames. 

4. The method of claim 3 wherein the step of 
15 producing the digitized control frame further comprises: 

recording the video images in the analog format 
outputted from the video camera on a video medium in an analog 
format ; 

playing back the video images recorded on the video 
2 0 medium for outputting the video images in the analog format; 

and 

wherein the step of producing the one or more 
digitized changed frames further comprises: 

recording the video images in the analog format 
25 outputted from the video camera on a video medium in an analog 

format; and 

playing back the video images recorded on the video 
medium for outputting the video images in the analog format. 

30 5. The method of claim 1 or 2 wherein the step of 

producing the digitized control frame further comprises: 

taking an image via a still frame camera of the body 
part in the control position to produce a still frame image in 
a digital format thereby producing the digitized control 

35 frame; and wherein the step of producing the digitized changed 

frame further comprises: 
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taking one or more images via a still frame camera 
of the body part in the one or more changed positions to 
produce a still frame image in a digital format thereby 
producing the one or more digitized changed frames. 

5 

6. The method of claim 1 or 2 wherein, prior to the 
step of comparing the digitized control frame with the 
digitized changed frame, the method further comprises: 

equalizing the digitized control frame to produce an 
10 equalized digitized control frame having an average intensity 

throughout and with a predetermined distribution of pixel 
intensity variations; 

equalizing the one or more digitized changed frames 
to produce one or more equalized digitized changed frames 
15 having an average intensity throughout and with a 

predetermined distribution of pixel intensity variations. 

7. The method of claim 1 or 2 wherein the step of 
comparing the digitized control frame with the digitized 

20 changed frame further comprises: 

subtracting out the common areas of pixel values 
between the digitized control frame and the one or more 
digitized changed frames to produce one or more digitized 
compare frames. 

25 

8. The method of claim 1 or 2 wherein the step of 
comparing the digitized control frame with the one or more 
digitized changed frames is further defined as producing one 
or more digitized compare frames, and wherein the step of 

3 0 comparing the digitized control frame and the one or more 

digitized changed frames further comprises: 

counting the pixels in the one or more digitized 

compare frames having a brightness value above a predetermined 

threshold level to provide the indication of shape changes 
35 over the defined surface area of the body part from the 

control position to the changed position. 



WO 02/03ROI 



PCT/US9 1 /05980 



36 

9. The method of claim 1 or 2 wherein the step of 
comparing the digitized control frame with the one or more 
digitized changed frames is further defined as producing one 
or more digitized compare frames, and wherein the step of 

5 comparing the digitized control frame and the one or more 

digitized changed frames further comprises: 

designating a control area of interest in each of 
the one or more digitized compare frames; and counting the 
pixels having a brightness value above a predetermined 
10 threshold level in the areas of interest - 

10. The method of claim 1 or 2 wherein the step of 
comparing the digitized control frame with the one or more 
digitized changed frames , is further defined as producing one 

15 or more digitized compare frames, and wherein the step of 

comparing the digitized control frame and the one or more 
digitized changed frames further comprises: 

designating a control area of interest in the 
digitized compare frame; 

2 0 designating a changed area of interest in the one or 

more digitized compare frames; and 

counting the pixels having a brightness value above 
a predetermined threshold level in the control level in the 
control area of interest and counting the pixels having a 

25 brightness value above a predetermined threshold level in the 

changed area of interest to provide the indication of shape 
changes over the defined surface area in the control area of 
interest with respect to the changed area of interest. 



30 11. The method of claim 1 or 2 wherein the step of 

comparing the digitized control frame with the one or more 
digitized changed frames further comprises producing one or 
more digitized compare frames, and wherein the step of 
comparing the digitized control frame and the one or more 

3 5 digitized changed frames further comprises: 

enhancing all of the pixels in each of the one or 



more digitized compare frames having a brightness level above 
a predetermined value whereby all of the pixels having a 
brightness value above a predetermined value appear to be more 
bright . 

12. The method of claim 1 or 2 wherein the step of 
comparing the digitized control frame with the one or more 
digitized changed frames is further defined as producing one 
or more digitized compare frames, and wherein the step of 
comparing the digitized control frame and the one or more 
digitized changed frames further comprises: 

replacing all of the pixels in each of the one or 
more digitized compare frames having a brightness value below 
a predetermined threshold level with a substantially null 
value and enhancing the pixels in each of the one or more 
digitized compare frames having a brightness value above a 
predetermined threshold level whereby the pixels in each of 
the one or more digitized compare frames having a brightness 
value above the predetermined threshold level appear more 
bright . 

13 . The method of claim 1 or 2 wherein the body 
part is further defined as an individual's face. 

14. The method of claim 13 further comprising: 
performing via an individual at least one of the 

following movements: (1) raising eyebrows from a rest 
position designated as the reference position to a raised 
position designated as the moved position, (2) smiling from a 
rest position designated as the reference position to a smiled 
position designated as the moved position; (3) closing the 
eyes from a rest position designated as the reference position 
to a fully closed position of the eyes designated as the moved 
position; and (4) wrinkling the nose from a rest position 
designated as the reference position to a fully wrinkled 
position designated as the moved position, and then producing 
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the one or more digitized control frame and the digitized 
changed frames. 

15. An apparatus for detecting motion of a body 
part comprising: 

means for producing a digitized frame of the body 
part in a reference position of the body part, identified as 
a digitized control frame; 

means for producing one or more digitized frames of 
the body part in a moved position of the body part identified 
as one or more digitized changed frames; and 

means for comparing the digitized control frame with 
each of the digitized changed frames and determining shape 
changes over a defined surface area of the body part from the 
control position to the moved position. 

16. The apparatus of claim 15 wherein the means for 
producing one or more digitized changed frames comprises: 

means for producing a plurality of digitized frames 

2 0 of the body part as the body part is moved from the reference 
position to the moved position and means for designating the 
digitized frames as digitized changed frames; and 

means for comparing each digitized changed frame 
with the digitized control frame to determine a parameter for 
25 each comparison with each movement parameter being indicative 

of motion of the body part between the digitized control frame 
and the digitized changed frame compared with the digitized 
control frame. 

3 0 17. The apparatus of claim 15 or 16 wherein the 
means for producing the digitized control frame further 
comprises : 

means for videoing using a video camera the body 
part to produce video images of the body part in an analog 
35 format; 

means for grabbing a frame of the video images in 



10 



39 

the analog format showing the body part in the control 
position; and 

means for digitizing the frame of the video image in 
the analog format and in the control position to produce the 
digitized control frame; and wherein the step of producing the 
one or more digitized changed frames further comprises: 

means for videoing using a video camera the body 
part to produce video images of the body part in an analog 
format ; 

means for grabbing one or more frames of the video 
images in the analog format showing the body part in one or 
more changed positions; and 

means for digitizing the one or more frames of the 
video image in the analog format and in the one -or more 
changed positions to produce the one or more digitized control 
frames . 

18, The apparatus of claim 17 wherein the means for 
producing the digitized control frame further comprises: 

means for recording the video images in the analog 
format outputted from the video camera on a video medium in an 

analog format; 

means for playing back the video images recorded on 
the video medium for outputting the video images in the analog 
format ; and 

wherein the means for producing the digitized 
changed frame further comprises: 

recording the video images in the analog format 
outputted from the video camera on a video medium in an analog 
format ; and 

means for playing back the video images recorded on 
the video medium for outputting. the video images in the analog 
format . 

19 . The apparatus of claim 15 or 16 wherein the 
means for producing the digitized control frame further 
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comprises : 

means for taking a image via a still frame camera of 
the body part in the control position to produce a still frame 
image in a digital format thereby producing the digitized 
5 control frame; and wherein the means for producing the 

digitized changed frame further comprises: 

means for taking one or more images via a still 
frame camera of the body part in the one or more changed 
positions to produce a still frame image in a digital format 
10 thereby producing the one or more digitized changed frames, 

20. The apparatus of claim 15 or 16 wherein said 
apparatus, the method further comprises: 

means for equalizing the digitized control frame 
15 prior to the step of comparing the digitized control frame 

with the digitized changed frame, to produce an equalized 
digitized control frame having an average intensity throughout 
and with a predetermined distribution of pixel intensity 
variations ; and 

2 0 means for equalizing the one or more digitized 

changed frames to produce one or more equalized digitized 
changed frames having an average intensity throughout and with 
a predetermined distribution of pixel intensity variations. 

25 21. The apparatus of claim 15 or 16 wherein the 

means for comparing the digitized control frame with the 

digitized changed frame further comprises: 

means for subtracting out the common areas of pixel 

values between the digitized control frame and the one or more 
30 digitized changed frames to produce one or more digitized 

compare frames. 

22. The apparatus of claim 15 or 16 wherein the 
means for comparing the digitized control frame with the one 
3 5 or more digitized changed frames is further defined as 

producing one or more digitized compare frames, and wherein 
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the means for comparing the digitized control frame and the 
one or more digitized changed frames further comprises: 

means for counting the pixels in the one or more 
digitized compare frames having a brightness value above a 
5 predetermined threshold level to provide the indication of 

shape changes over the defined surface area of the body part 
from the control position to the changed position. 

23. The apparatus of claim 15 or 16 wherein the 
L0 means for comparing the digitized control frame with the one 
or more digitized changed frames further includes means for 
producing one or more digitized compare frames, and wherein 
the means for comparing the digitized control frame and the 
one or more digitized changed frames further comprises: 
15 means for designating a control area of interest in 

each of the one or more digitized compare frames; and means 
for counting the pixels having a brightness value above a 
predetermined threshold level in the areas of interest. 

20 24. The apparatus of claim 15 or 16 wherein the 

means for comparing the digitized control frame with the one 
or more digitized changed frames, further comprises means for 
producing one or more digitized compare frames, and wherein 
the means for comparing the digitized control frame and the 

25 one or more digitized changed frames further comprises: 

means for designating a control area of interest in 
the digitized compare frame; 

means for designating a changed area of interest in 
the one or more digitized compare frames; and 

30 means for counting the pixels having a brightness 

value above a predetermined threshold level in the control 
level in the control area of interest and means for counting 
the pixels having a brightness value above a predetermined 
threshold level in the changed area of interest to provide the 

35 indication of shape changes over the defined surface area xn 

the control area of interest with respect to the changed area 
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of interest. 

25. The apparatus of claim 15 or 16 wherein the 
means for comparing the digitized control frame with the one 
5 or more digitized changed frames is further defined as 

producing one or more digitized compare frames, and wherein 
the means for comparing the digitized control frame and the 
one or more digitized changed frames further comprises: 

means for enhancing all of the pixels in each of the 
10 one or more digitized compare frames having a brightness level 

above a predetermined value whereby all of the pixels having 
a brightness value above a predetermined value appear to be 
more bright. 



15 26. The apparatus of claim 15 or 16 wherein the 

means for comparing the digitized control frame with the one 
or more digitized changed frames further comprises means for 
producing one or more digitized compare frames, and wherein 
the means for comparing the digitized control frame and the 

20 one or more digitized changed frames further comprises: 

means for replacing all of the pixels in each of the 
one or more digitized compare frames having a brightness value 
below a predetermined threshold level with a substantially 
null value and means for enhancing the pixels in each of the 

25 one or more digitized compare frames having a brightness value 

above a predetermined threshold level whereby the pixels in 
each of the one or more digitized compare frames having a 
brightness value above the predetermined threshold level 
appear more bright. 
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